Skip to content

feat: adds basic SQL prefilter for bid screening requests#3184

Merged
stalniy merged 2 commits into
mainfrom
feat/db-filter
May 18, 2026
Merged

feat: adds basic SQL prefilter for bid screening requests#3184
stalniy merged 2 commits into
mainfrom
feat/db-filter

Conversation

@stalniy
Copy link
Copy Markdown
Contributor

@stalniy stalniy commented May 15, 2026

Why

ref CON-341

What

Summary by CodeRabbit

  • New Features

    • Candidate-focused bid screening with aggregated criteria for more precise matching.
  • Bug Fixes

    • Stricter attribute-key validation and enforcement that persistent storage volumes declare a non‑RAM storage class.
  • Refactor

    • Screening flow reworked to fetch/evaluate candidates via a dedicated repository; service interfaces updated and legacy provider methods removed.
  • Tests

    • Added/expanded unit and integration tests for aggregation, screening, and cluster-state hydration; removed some legacy provider tests.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 15, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR tightens bid-screening schemas, adds cluster-state hydration and criteria aggregation, implements DB-side candidate querying in BidScreeningRepository, refactors the service to consume candidates, updates tests, and removes legacy provider-inventory methods.

Changes

Bid-Screening Candidate Filtering Refactor

Layer / File(s) Summary
Persistent storage & attribute key schema validation
apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts, apps/provider-inventory/src/http-schemas/bid-screening.schema.ts
Attribute key now validated by SDL_ATTRIBUTE_KEY_REGEX (length and format); StorageResourceSchema adds .superRefine() requiring a non-empty class that is not "ram" when persistent: "true" is present, emitting a custom Zod issue on attributes.
Cluster state hydration
apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.ts, apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.spec.ts
New hydrateClusterState, hydrateNode, hydratePair convert raw inventory JSONB into typed ClusterState with ResourcePair bigints; tests validate bigint preservation and nullish input behavior.
Criteria aggregation
apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.ts, apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts
aggregateCriteria computes BigInt totals/maxima, partitions storage into persistent vs ephemeral (skips RAM), separates exact vs trailing-* glob attributes (regex-escaped), preserves signedBy, and scaffolds per-unit filters; tests cover totals, maxima, attribute partitioning, signedBy, and units.
BidScreeningRepository database queries
apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.ts, apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts
New BidScreeningRepository.findCandidates builds Drizzle SQL from aggregated criteria (online/uptime constraints, capacity minima, per-replica free limits, optional JSON containment, glob-pattern self-attribute matching, audited-by allOf/anyOf), hydrates cluster JSONB and computes isAudited; exports AUDITOR and BidScreeningCandidate type.
Repository integration tests
apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts
Integration tests validate capacity filtering, signed-by semantics (allOf/anyOf and empty omission), exact/glob self-attribute matching (path-separator rules), case-sensitive comparisons, isAudited projection, online constraints, and JSONB -> ResourcePair hydration; includes seeding helpers.
BidScreeningService refactor
apps/provider-inventory/src/services/bid-screening/bid-screening.service.ts, apps/provider-inventory/src/services/bid-screening/bid-screening.service.spec.ts
Service now aggregates resource units, calls BidScreeningRepository.findCandidates, filters candidates with matcher.match(candidate.cluster, resourceUnits), and returns results built from candidate fields including isAudited; tests updated to the candidate-based flow and setup/mocks replaced.
Cleanup of legacy provider-inventory methods
apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts, apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
Removed getOnlineProviders() and getAuditedProviderAddresses() and their hydration utilities/imports; deleted corresponding test suites now covered by new repository tests.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • akash-network/console#3171: Both PRs refactor the bid-screening flow by changing apps/provider-inventory/src/services/bid-screening/bid-screening.service.ts (constructor/dependencies and how matching results are produced).
  • akash-network/console#3172: Overlaps on bid-screening service refactor and candidate vs provider-inventory matching responsibilities.

Suggested reviewers

  • ygrishajev
  • baktun14
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/db-filter

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 15, 2026

Codecov Report

❌ Patch coverage is 90.32258% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 62.94%. Comparing base (e1d2bba) to head (1d960b1).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...inventory/src/http-schemas/bid-screening.schema.ts 0.00% 6 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3184      +/-   ##
==========================================
- Coverage   63.69%   62.94%   -0.76%     
==========================================
  Files        1089     1050      -39     
  Lines       26433    25431    -1002     
  Branches     6408     6248     -160     
==========================================
- Hits        16837    16008     -829     
+ Misses       8400     8238     -162     
+ Partials     1196     1185      -11     
Flag Coverage Δ *Carryforward flag
api 84.24% <ø> (-0.06%) ⬇️
deploy-web 46.68% <ø> (ø) Carriedforward from e1d2bba
log-collector ?
notifications 91.06% <ø> (ø) Carriedforward from e1d2bba
provider-console 81.48% <ø> (ø) Carriedforward from e1d2bba
provider-inventory 81.88% <90.32%> (+0.53%) ⬆️
provider-proxy 86.08% <ø> (ø) Carriedforward from e1d2bba
tx-signer ?

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
...lib/hydrate-cluster-state/hydrate-cluster-state.ts 100.00% <100.00%> (ø)
...sitories/bid-screening/bid-screening.aggregator.ts 100.00% <100.00%> (ø)
...sitories/bid-screening/bid-screening.repository.ts 100.00% <100.00%> (ø)
...rovider-inventory/provider-inventory.repository.ts 71.42% <ø> (-15.24%) ⬇️
...rc/services/bid-screening/bid-screening.service.ts 100.00% <100.00%> (+5.71%) ⬆️
...inventory/src/http-schemas/bid-screening.schema.ts 0.00% <0.00%> (ø)

... and 44 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts`:
- Around line 14-31: The StorageResourceSchema currently allows any
attributes[].key but must enforce the same provider-inventory SDL key format;
update validation so AttributeSchema.key (or the attributes array validation
inside StorageResourceSchema) matches the provider-inventory SDL key pattern
(use the exact regex/validator used in provider-inventory) and return Zod issues
on the specific attribute key (path like ["attributes", i, "key"]) with a clear
message when it fails; ensure the change reuses AttributeSchema if possible so
other schemas stay consistent and preserve OpenAPI metadata.

In
`@apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.ts`:
- Around line 16-43: The hydration functions assume well-formed input and can
throw when raw is malformed; make hydrateClusterState, hydrateNode and
hydratePair defensive: ensure cluster.nodes is an array before mapping (use
Array.isArray(cluster.nodes) ? cluster.nodes : []), ensure cluster.storage is a
plain object before Object.entries (fallback to {}), and in hydrateNode guard
node.gpu and node.gpu.quantity (use node.gpu?.quantity ?? 0 and node.gpu?.info ?
node.gpu.info : []) and default fields like storageClasses/cpus to [] when not
arrays; replace direct BigInt conversions in hydratePair with a safe conversion
helper (e.g., coerce to number/string then try/catch BigInt or default to 0) so
BigInt(pair.allocatable) and BigInt(pair.allocated) never throw on malformed
inputs.

In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts`:
- Line 12: Replace the relative import of the repository with the backend source
path alias: change the import line that currently brings in AUDITOR and
BidScreeningRepository from "./bid-screening.repository" to use the `@src/`* alias
(e.g., import { AUDITOR, BidScreeningRepository } from
"@src/repositories/bid-screening/bid-screening.repository"), updating the module
specifier so the test uses the aliased backend source path.

In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.ts`:
- Line 11: Replace the relative import of aggregateCriteria and
BidScreeningCriteria from "./bid-screening.aggregator" with the backend source
path alias import using `@src` so the file uses the project aliasing convention;
update the import statement that references aggregateCriteria and type
BidScreeningCriteria in bid-screening.repository.ts to import from the
corresponding module via the `@src/`* alias (keeping the same exported symbols and
module name bid-screening.aggregator).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d2a5fb9f-7083-48c2-823a-43a914f5396a

📥 Commits

Reviewing files that changed from the base of the PR and between 67d7c04 and b5b5268.

📒 Files selected for processing (13)
  • apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.spec.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.spec.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.ts
💤 Files with no reviewable changes (2)
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts

Comment thread apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts (2)

5-5: ⚡ Quick win

Use backend source alias import instead of relative path.

Switch Line 5 to @src/* alias to match backend import rules.

Suggested fix
-import { aggregateCriteria } from "./bid-screening.aggregator";
+import { aggregateCriteria } from "@src/repositories/bid-screening/bid-screening.aggregator";

As per coding guidelines "apps/**/*.ts: Use path aliases @src/* for source files and @test/* for test files in backend applications".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts`
at line 5, The test import currently uses a relative path; update the import in
bid-screening.aggregator.spec.ts so it uses the backend source alias (e.g.,
replace the relative import of aggregateCriteria from
"./bid-screening.aggregator" with the `@src/`* alias equivalent) to conform to
backend rules; ensure the import still references the same exported symbol
aggregateCriteria and that any test runner/tsconfig path mappings already
support the `@src` alias.

26-31: ⚡ Quick win

Add max-per-replica assertions for the count=0 case.

This test already defines the edge case; asserting maxPerReplicaCpu/Memory/Gpu stay 0n would lock in correct behavior and catch regressions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts`
around lines 26 - 31, The test "count=0 contributes nothing to totals" should
also assert that per-replica maxima remain zero: update the spec that calls
aggregateCriteria([makeUnit({... count: 0 })], makeRequirements()) to include
expect(c.maxPerReplicaCpu).toBe(0n), expect(c.maxPerReplicaMemory).toBe(0n) and
expect(c.maxPerReplicaGpu).toBe(0n); this ensures aggregateCriteria's
max-per-replica behavior for the zero-count unit (referenced symbols:
aggregateCriteria, makeUnit, makeRequirements) is locked in and will catch
regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.ts`:
- Around line 46-48: The max-per-replica variables (maxPerReplicaCpu,
maxPerReplicaMemory, maxPerReplicaGpu) are being updated even for entries with
count === 0; change the comparisons so they only update when the item’s count is
greater than zero (e.g., guard the existing checks with count > 0), so use
conditions like if (count > 0 && cpu > maxPerReplicaCpu) and similarly for
memory and gpu to avoid zero-count entries inflating the maxima in the
aggregation logic in bid-screening.aggregator (variables: cpu, memory, gpu,
count, maxPerReplicaCpu, maxPerReplicaMemory, maxPerReplicaGpu).

---

Nitpick comments:
In
`@apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts`:
- Line 5: The test import currently uses a relative path; update the import in
bid-screening.aggregator.spec.ts so it uses the backend source alias (e.g.,
replace the relative import of aggregateCriteria from
"./bid-screening.aggregator" with the `@src/`* alias equivalent) to conform to
backend rules; ensure the import still references the same exported symbol
aggregateCriteria and that any test runner/tsconfig path mappings already
support the `@src` alias.
- Around line 26-31: The test "count=0 contributes nothing to totals" should
also assert that per-replica maxima remain zero: update the spec that calls
aggregateCriteria([makeUnit({... count: 0 })], makeRequirements()) to include
expect(c.maxPerReplicaCpu).toBe(0n), expect(c.maxPerReplicaMemory).toBe(0n) and
expect(c.maxPerReplicaGpu).toBe(0n); this ensures aggregateCriteria's
max-per-replica behavior for the zero-count unit (referenced symbols:
aggregateCriteria, makeUnit, makeRequirements) is locked in and will catch
regressions.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ab3adc42-42bc-4eca-9d2a-e1ec2f75efc4

📥 Commits

Reviewing files that changed from the base of the PR and between b5b5268 and 2e81d3e.

📒 Files selected for processing (13)
  • apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.spec.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.spec.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.aggregator.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.spec.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.ts
💤 Files with no reviewable changes (2)
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.integration.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.spec.ts
🚧 Files skipped from review as they are similar to previous changes (9)
  • apps/api/src/bid-screening/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/http-schemas/bid-screening.schema.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.ts
  • apps/provider-inventory/src/lib/hydrate-cluster-state/hydrate-cluster-state.spec.ts
  • apps/provider-inventory/src/repositories/provider-inventory/provider-inventory.repository.ts
  • apps/provider-inventory/src/services/bid-screening/bid-screening.service.spec.ts
  • apps/provider-inventory/src/repositories/bid-screening/bid-screening.repository.integration.ts

@stalniy stalniy force-pushed the feat/db-filter branch 2 times, most recently from 2a05397 to 3dd07f6 Compare May 15, 2026 14:19
@baktun14
Copy link
Copy Markdown
Contributor

question: I noticed ip leases filters providers quite significantly, is this taken into account?

@stalniy stalniy added this pull request to the merge queue May 18, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks May 18, 2026
@stalniy stalniy added this pull request to the merge queue May 18, 2026
Merged via the queue into main with commit 0be6b24 May 18, 2026
56 checks passed
@stalniy stalniy deleted the feat/db-filter branch May 18, 2026 03:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants